On the Excess Entropy of a Mixture of Sources and the Average Redundancy Rate of Adaptive Block Codes
نویسندگان
چکیده
It is well known that the average redundancy rate of sample-based universal block codes is monotonically decreasing with the length of samples used for their construction. We show, that this is no longer true if samples and blocks of symbols to be compressed are produced by two different sources. In this case, the formula for the average redundancy receives an additional term which changes the overall dependency on the sample length `, such that there exists a point ` = `∗, where the redundancy is minimal. This optimal sample size is asymptotically: `∗ = m−1 2 D(T ||S) + O ( 1 n ) , where m is a cardinality of the alphabet, n is a block size, and D (T ||S) is the relative entropy between the sources T , S producing samples and input data correspondingly. We use this finding to show how to design optimal sequential and random-accesscapable compression systems based on adaptive block codes.
منابع مشابه
Redundancy-Related Bounds on Generalized Huffman Codes
This paper presents new lower and upper bounds for the compression rate of optimal binary prefix codes on memoryless sources according to various nonlinear codeword length objectives. Like the most well-known redundancy bounds for minimum (arithmetic) average redundancy coding — Huffman coding — these are in terms of a form of entropy and/or the probability of the most probable input symbol. Th...
متن کاملOn the redundancy of Slepian--Wolf coding
In this paper, the redundancy of both variable and fixed rate Slepian–Wolf coding is considered. Given any jointly memoryless source-side information pair with finite alphabet, the redundancy of variable rate Slepian–Wolf coding of with decoder only side information depends on both the block length and the decoding block error probability , and is defined as the difference between the minimum...
متن کاملImproved Redundancy Bounds for Exponential Objectives
We present new lower and upper bounds for the compression rate of binary prefix codes optimized over memoryless sources according to two related exponential codeword length objectives. The objectives explored here are exponential-average length and exponential-average redundancy. The first of these relates to various problems involving queueing, uncertainty, and lossless communications, and it ...
متن کاملUniversal noiseless coding
A&ruct-Universal coding is any asymptotically opt imum method of block-to-block memoryless source coding for sources with unknown parameters. This paper considers noiseless coding for such sources, primarily in terms of variable-length coding, with performance measured as a function of the coding redundancy relative to the per-letter conditional source entropy given the unknown parameter. It is...
متن کاملBlock-Based Compressive Sensing Using Soft Thresholding of Adaptive Transform Coefficients
Compressive sampling (CS) is a new technique for simultaneous sampling and compression of signals in which the sampling rate can be very small under certain conditions. Due to the limited number of samples, image reconstruction based on CS samples is a challenging task. Most of the existing CS image reconstruction methods have a high computational complexity as they are applied on the entire im...
متن کامل